Critical periods when dopamine controls behavioral responding during Pavlovian learning.

RATIONALE: Learning the association between rewards and predictive cues is critical for appetitive behavioral responding. The mesolimbic dopamine system is thought to play an integral role in establishing these cue-reward associations. The dopamine response to cues can signal differences in reward value, though this emerges only after significant training. This suggests that the dopamine system may differentially regulate behavioral responding depending on the phase of training. OBJECTIVES: The purpose of this study was to determine whether antagonizing dopamine receptors elicited different effects on behavior depending on the phase of training or the type of Pavlovian task. METHODS: Separate groups of male rats were trained on Pavlovian tasks in which distinct audio cues signaled either differences in reward size or differences in reward rate. The dopamine receptor antagonist flupenthixol was systemically administered prior to either the first ten sessions of training (acquisition phase) or the second ten sessions of training (expression phase), and we monitored the effect of these manipulations for an additional ten training sessions. RESULTS: We identified acute effects of dopamine receptor antagonism on conditioned responding, the latency to respond, and post-reward head entries in both Pavlovian tasks. Interestingly, dopamine receptor antagonism during the expression phase produced persistent deficits in behavioral responding only in rats trained on the reward size Pavlovian task. CONCLUSIONS: Together, our results illustrate that dopamine's control over behavior in Pavlovian tasks depends upon one's prior training experience and the information signaled by the cues.

Sex Differences in Behavioral Responding and Dopamine Release during Pavlovian Learning.

Learning associations between cues and rewards require the mesolimbic dopamine system. The dopamine response to cues signals differences in reward value in well trained animals. However, these value-related dopamine responses are absent during early training sessions when cues signal differences in the reward rate. These findings suggest cue-evoked dopamine release conveys differences between outcomes only after extensive training, though it is unclear whether this is unique to when cues signal differences in reward rate, or whether this is also evident when cues signal differences in other value-related parameters such as reward size. To address this, we used a Pavlovian conditioning task in which one audio cue was associated with a small reward (one pellet) and another audio cue was associated with a large reward (three pellets). We performed fast-scan cyclic voltammetry to record changes in dopamine release in the nucleus accumbens of male and female rats throughout learning. While female rats exhibited higher levels of conditioned responding, a faster latency to respond, and elevated post-reward head entries relative to male rats, there were no sex differences in the dopamine response to cues. Multiple training sessions were required before cue-evoked dopamine release signaled differences in reward size. Reward-evoked dopamine release scaled with reward size, though females displayed lower reward-evoked dopamine responses relative to males. Conditioned responding related to the decrease in the peak reward-evoked dopamine response and not to cue-evoked dopamine release. Collectively, these data illustrate sex differences in behavioral responding as well as in reward-evoked dopamine release during Pavlovian learning.

Delays to Reward Delivery Enhance the Preference for an Initially Less Desirable Option: Role for the Basolateral Amygdala and Retrosplenial Cortex.

Temporal costs influence reward-based decisions. This is commonly studied in temporal discounting tasks that involve choosing between cues signaling an imminent reward option or a delayed reward option. However, it is unclear whether the temporal delay before a reward can alter the value of that option. To address this, we identified the relative preference between different flavored rewards during a free-feeding test using male and female rats. Animals underwent training where either the initial preferred or the initial less preferred reward was delivered noncontingently. By manipulating the intertrial interval during training sessions, we could determine whether temporal delays impact reward preference in a subsequent free-feeding test. Rats maintained their initial preference if the same delays were used across all training sessions. When the initial less preferred option was delivered after short delays (high reward rate) and the initial preferred option was delivered after long delays (low reward rate), rats expectedly increased their preference for the initial less desirable option. However, rats also increased their preference for the initial less desirable option under the opposite training contingencies: delivering the initial less preferred reward after long delays and the initial preferred reward after short delays. These data suggest that sunk temporal costs enhance the preference for a less desirable reward option. Pharmacological and lesion experiments were performed to identify the neural systems responsible for this behavioral phenomenon. Our findings demonstrate the basolateral amygdala and retrosplenial cortex are required for temporal delays to enhance the preference for an initially less desirable reward.SIGNIFICANCE STATEMENT The goal of this study was to determine how temporal delays influence reward preference. We demonstrate that delivering an initially less desirable reward after long delays subsequently increases the consumption and preference for that reward. Furthermore, we identified the basolateral amygdala and the retrosplenial cortex as essential nuclei for mediating the change in reward preference elicited by sunk temporal costs.

Dopamine release and its control over early Pavlovian learning differs between the NAc core and medial NAc shell.

Dopamine neurons respond to cues to reflect the value of associated outcomes. These cue-evoked dopamine responses can encode the relative rate of reward in rats with extensive Pavlovian training. Specifically, a cue that always follows the previous reward by a short delay (high reward rate) evokes a larger dopamine response in the nucleus accumbens (NAc) core relative to a distinct cue that always follows the prior reward by a long delay (low reward rate). However, it was unclear if these reward rate dopamine signals are evident during early Pavlovian training sessions and across NAc subregions. To address this, we performed fast-scan cyclic voltammetry recordings of dopamine levels to track the pattern of cue- and reward-evoked dopamine signals in the NAc core and medial NAc shell. We identified regional differences in the progression of cue-evoked dopamine signals across training. However, the dopamine response to cues did not reflect the reward rate in either the NAc core or the medial NAc shell during early training sessions. Pharmacological experiments found that dopamine-sensitive conditioned responding emerged in the NAc core before the medial NAc shell. Together, these findings illustrate regional differences in NAc dopamine release and its control over behavior during early Pavlovian learning.

Cocaine experience abolishes the motivation suppressing effect of CRF in the ventral midbrain.

Stress affects dopamine-dependent behaviors in part through the actions of corticotropin releasing factor (CRF) in the ventral tegmental area (VTA). For example, acute stress engages CRF signaling in the VTA to suppress the motivation to work for food rewards. In contrast, acute stress promotes drug-seeking behavior through the actions of CRF in the VTA. These diverging behavioral effects in food- and drug-based tasks could indicate that CRF modulates goal-directed actions in a reinforcer-specific manner. Alternatively, prior drug experience could functionally alter how CRF in the VTA regulates dopamine-dependent behavior. To address these possibilities, we examined how intra-VTA injections of CRF influenced cocaine intake and whether prior drug experience alters how CRF modulates the motivation for food rewards. Our results demonstrate that intra-VTA injections of CRF had no effect on drug intake when self-administering cocaine under a progressive ratio reinforcement schedule. We also found that a prior history of either contingent or noncontingent cocaine infusions abolished the capacity for CRF to reduce the motivation for food rewards. Furthermore, voltammetry recordings in the nucleus accumbens illustrate that CRF in the VTA had no effect on cocaine-evoked dopamine release. These results collectively illustrate that exposure to abused substances functionally alters how neuropeptides act within the VTA to influence motivated behavior.

Dopamine Encodes Retrospective Temporal Information in a Context-Independent Manner.

The dopamine system responds to reward-predictive cues to reflect a prospective estimation of reward value, although its role in encoding retrospective reward-related information is unclear. We report that cue-evoked dopamine release in the nucleus accumbens core encodes the time elapsed since the previous reward or rather the wait time. Specifically, a cue that always follows the preceding reward with a short wait time elicits a greater dopamine response relative to a distinct cue that always follows the preceding reward with a long wait time. Differences in the dopamine response between short wait and long wait cues were evident even when these cues were never experienced together within the same context. Conditioned responding updated accordingly with a change in cue-evoked dopamine release but was unrelated to a difference in the dopamine response between cues. Collectively, these findings illustrate that the cue-evoked dopamine response conveys a subjective estimation of the relative reward rate.